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INTRODUCTION. 


p . 


The  accomplishments  of  the  work  performed  with  whole  or  partial 
support  of  the  Grant  AFOSR  72-2371  are  outlined  in  Section  I.  The  papers 
published  with  grant  support  are  given  in  Section  II  along  with  those  that 
are  in  some  prepublication  stage.  Section  III  lists  the  technical  personnel 
supported  by  the  grant  during  each  year  period. 
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SECTION  I.  ACCOMPLLSMc'.ENTS. 

I 

The  main  grant  research  effort  has  been  in  tlie  area  of  nonpara- 
metric  discrimination  and  the  closely  related  problem  of  density  estimation. 
In  the  discrimination  problem  a statistician  makes  an  observation  X,  a 
random  vector  with  values  in  , and  wishes  to  estimate  its  state  e,  a 
random  variable  taking  values  in  {1,...,M},  All  that  he  knows  about  the 
distribution  of  (X,e)  is  that  which  can  be  inferred  from  a sample  (X.j,0.j), 

, drawn  from  that  distribution.  The  sample,  commonly 

called  data,  is  assumed  to  be  independent  of  (X,e).  Using  X and  the  data 

A 

the  statistician  makes  a randomized  decision  0 for  0 where  his  rule  is  any 
: procedure  which  determines  the  probability  distribution  6 = for 

A 

0 given  X and  the  data.  In  particular, 

6:  IR^  X (IR^^  X M})" [0,1]”  , 

= 1 , and 

Pie  = j|x,(x^,o^),...,(x^,e^)}  = 6^.  . 1 < j < m 

For  his  data  and  rule,  the  probability  of  error  is 
= Pie  f e|(Xp6^),...,(x^,e^)}  , 

a random  variable  whose  value  is  the  limiting  frequency  of  errors  made  when 
a large  number  of  independent  observations  have  their  states  estimates  with 
6 and  the  given  data. 

The  questions  which  have  been  considered  in  the  past  usually  are 
of  the  asymptotic  type.  For  example,  what  does  converge  to  as  n tends 
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to  infinity  and  if,  say,  ■?  L in  probability,  how  does  L compare  to  the 
Bayes  probability  of' error  L*.  Other  asyn)f>totic  studies  have  been  con- 
cerned with  how  one  estimates  L,  or  L*  if  it  is  different  from  L,  from  the 
data.  Investigators  have  also  been  recently  concerned  with  estimating 
from  the  data,  sometimes  called  error  estimation.  If  is  a particular 
estimate  of  the  statistician  would  like  to  know  how  small 

P[iL„-LJ>e]  (1) 

is  for  a given  e > 0.  The  difficulty  is  that  the  distribution  of  (X,0) 
is  unknown  and  the  best  that  one  can  hope  for  is  a distribution- free  upper- 
bound  for  (1)  which  depends  only  on  n,  e and  d and  which  tends  to  zero  with 
n for  each  fixed  e and  d. 

With  grant  support,  a summary  of  all  recent  work  on  nonparametric 
discrimination,  including  density  estimation,  was  completed  [12].  In  [20, 

41]  a class  of  rules,  which  are  a natural  generalization  of  k-nearest  neighbor 
rules  and  which  are  called  voting  rules,  are  discussed  and  conditions  are 
given  for  which  -2  L*  in  probability  and  with  probability  one.  In  [19, 

43],  the  asymptotic  properties  of  k-nearest  neighbor  rules  are  investigated 
for  the  nonparametric  estimation  problem  (0  now  takes  values  in  IR^  rather 
than  in  {1 , . . . ,M }) . 

A consideration  always  present  in  nonparametric  discrimination 
is  how  to  implement  rules  derived  from  large  amounts  of  data.  For  example, 
if  one  uses  the  k-nearest  neighbor  rule  with  the  data,  a large  n presents 
difficulties  in  that  both  storage  requirements  and  computation  times  increase 
with  n.  In  order  to  keep  the  implementation  requirements  within  reason  and 
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still  retain  the  appeal  of  k-nearesl.  neighbor  rules,  various  procedures 

I 

for  condensing  or  editing  tiie  data  before  the  k-nearest  neighbor  rule  is 
applied  have  been  suggested.  In  [5],  a simple  argument  for  the  asymptotic 
property  of  one  of  the  most  appealing  of  these  schemes  is  given  while  in 
[23],  a basic  flaw  in  the  original  argument  used  for  obtaining  the  asymptotic 
properties  of  the  edited  nearest  neighbor  rule  is  pointed  out. 

In  error  estimation,  the  first  distribution-free  bound  for  (1) 
was  given  for  k-local  rules  in  [29]^  where  is  the  deleted  estimate  of 
A rule  is  k^local  if  the  decision  e depends  only  on  X and  the  pairs 
(X^,0^-)  for  which  X^  is  one  of  the  k-closest  to  X from  X^...,X^.  The  bound 
for  (1)  given  in  [29]  is  of  the  form  A/ne  where  A is  an  explicitly  given 
small  constant  depending  only  on  M and  k.  Thus  deleted  estimates  appear 
to  be  very  good  estimates  of  when  k-local  rules  are  used.  This  has  also 
been  confirmed  in  the  extensive  simulation  [39]  which  shows  that  the  bound 
of  [29]  is  pessimistic.  The  deleted  estimate  has  also  been  shown  to  be  an 
asymptotically  consistent  estimate  of  the  Bayes  risk  L*  for  a wide  variety 
of  rules  [2].  Using  the  resubstitution  estimate  of  an  exponential 
bound  for  (1)  has  been  found  for  linear  rules  and  condensed  nearest  neighbor 
rules  [17,22,26,38].  Bounds  of  the  type  A/»^  have  also  been  obtained  for 
(1)  for  the  popular  class  of  two-step  rules  [42]. 

^ Ironically,  this  first  and,  in  some  ways,  best  paper  that  I've  been  in- 
volved in  for  this  area  has  had  to  undergo  several  revisions  with  long 
editorial  delays  between  them.  The  third  revision  is  now  being  looked  at. 
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In  nonparamotric  density  estimation  one  seeks  to  estimate  the 
density  f of  the  sample  Tv/o  popular  estimates  have  been  investi- 

gated under  this  grant.  The  kernel  estimate  f^  is  given  by 


= E K((x-X^)/h^)/nh|^‘^ 


(2) 


where  K,  the  kernel,  is  a probability  density  on  IR  and  {h^^l  is  a sequence 


of  positive  numbers  tending  to  zero  with  n.  In  [33]  conditions  are  given 
which  insure  that 


fpCx)  ->  f(x)  w.p.l 


sup  |f„(x)  - f(x)|  ->  0 w.p.l 


(3) 

(4) 


In  both  cases,  these  conditions  are  weaker  than  any  in  the  literature. 
Additionally,  in  [34]  conditions  are  given  which  insure 


(5) 


Similar  types  of  results  are  developed  in  [36].  The  sequence  {h^}  in  (2) 

Is  chosen  without  regard  to  X^ something  which  one  would  like  to 

be  able  to  do.  In  [11]  conditions  are  given  which  allow  h^  = h^(X^ ,. . . .X^^) 
while  still  yielding  (3)  and  (4).  These  results  are  extended  in  [40], 

The  nearest  neighbor  estimate  of  f is  given  by 


k /n 

g„(x)  = 4- 


where  {k  } is  a sequence  of  positive  integers  with  k < n and 


k^/n 

I 

kn 

I 

and  is  the  volume  oi  the  smallest  sphere,  centered  at  x,  which  contains 
of  the  points  In  [3]  conditions  are  given  for 

g^Cx)  ^ f(x)  w.p.1 

while  in  [22]  conditions  are  given  for 

sup  |g  (x)  - f(x)|  0 w.p.l  . 

X 

Other  results  related  to  density  estimation  may  be  found  in  [4,31,32,35] 
while  [27]  contains  a density  estimation  result  which  applies  directly  to 
the  clustering  problem. 

Random  search  deals  with  the  problem  of  locating  the  minimum  of 
an  unknown  function  g defined  on  IR^.  The  function  g is  not  assumed  to 
possess  any  of  the  usual  analytical  properties,  such  as  convexity,  that 
are  invoked  when  one  is  searching  for  extrema.  For  this  purpose  one  gene- 
rates a sequence  - of  random  vectors  with  values  in  IR^  and  with 

representing  the  estimate  of  the  location  of  a global  minimum  after  n steps. 
The  goal  is  to  produce  a sequence  for  which  g(Z^)  converges  with  probability 
one,  to  the  essential  infimum  of  g(x).  All  random  search  methods  generate 
a trial  vector  Z*^-|  from  Z^...,Z^  and  then  let 

■Z„  if  g(ZJ  <g(z;„) 

■ z;„  if  g(zj„)  : g(z„)  . 

A further  complication  can  arise  when,  for  any  x,  one  cannot  observe  g(x), 
but  only  a sample  with  distribution  function  and  mean  g(x). 
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} 


► 


[ 


If  r denotes  Ltio  distribution  function  of  X*  , given  X,  j 

n . n+ 1 1 n j 

and  g(x)  denotes  the'  sample  mean  of  a sample  of  a size  with  the  distri- 
bution function  then  Devroye  [13],  for  T 

’f  < i(X„)  - j 

X otheni'ise  , j 

**  i 

1 

I 

gave  conditions  on  F^,  (Cp) > fx^}  which  insure  that  ' 

g(X^)  5 ess  inf  g(x)  w.p.l  . ' 

Further  refinements  may  be  found  in  [14,18,21,30,37]. 

I 

The  grant  has  also  been  used  to  partially  support  several  applied  '1 

f i 

pattern  recognition  projects.  Under  the  direction  of  J.K.  Aggarwal , a t! 

flying  spot  scanner  system  with  color  capabilities  has  been  fabricated  and 

used  to  analyze  aerial  color  infrared  photographs  [24,28],  primarily  for  j 

i 

the  detection  of  diseases  in  citrus  trees.  Additionally,  a computer  analy- 
sis of  planar  curvilinear  moving  images  was  undertaken  in  [25]. 

Finally,  the  grant  has  supplied  partial  support  for  system  theory 
work  in  [6,7],  for  the  analysis  and  design  of  digital  filters  in  [1,8,10, 

15,16],  and  the  analysis  of  computer  storage  systems  in  [9]. 


f 
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SECTION  II.  PAPCNS  PUBLISHCD  WITH  ArOSR  GRANT  72-2371  SUPPORT. 


If  a paper  v/as  published  in  the  proceedings  of  a conference  and 
later  in  a refereed  journal,  only  the  listing  for  the  journal  publication 
is  given  below. 


[1]  E.P.F.  Kan  and  J.K.  Aggarwal , "Multirate  digital  filtering,"  JEl'^H 
Tmnsaations  on  Audio  and  Elec tvoaaouB tics,  AU-20,  223-225,  1972. 

[2]  T.O.  Wagner,  "Deleted  estimates  of  the  Bayes  risk,"  Annals  of  Statistics, 

I,  359-362,  1973. 

[3]  T.J.  Wagner,  "Strong  consistency  of  a nonparametric  estiniate  of  a 
density  function,"  IEEE  Transactions  on  Systems,  Nan  and  Cybernetics, 

SMC- 3.  289-290,  1973. 

[4]  T.J.  Wagner  and  C.K.  Chow,  "Consistency  of  an  estimate  of  tree-depen- 
dent probability  distributions,"  IEEE  Transactions  on  Information 
Theory,  IT-19,  369-371,  1973. 

[5]  T.J.  Wagner,  "Convergence  of  the  edited  nearest  neighbor,"  IEEE 
Transactions  on  Information  Theory,  IT-19,  696-697,  1973. 

[6]  J.K.  Aggarwal  and  D.H.  Eller,  "Optimization  of  functional  differen- 
tial systems,"  Journal  of  Optimization  Theory  and  Applications,  1 1 , 
100-120,  1973. 

[7]  J.K.  Aggarwal,  "Feedback  control  of  linear  systems  with  distributed 
delay,"  Automatica,  9,  367-379,  1973. 

[8]  J.K.  Aggarwal,  "Input  quantization  and  arithmetic  round-off  in  digital 
filters  - a review,"  network  and  Signal  Theory  (J.K.  Skwizynski  and 

J. O.  Scarilan,  Eds.)  Peter  Peregrinus  Ltd.,  London,  315-343,  1973. 
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[9]  T.J.  Wagner  and  P.A.  Franos^ckj  "SoiVie  distribution-free  aspects  of 

I 

paging  algoritinn  performance,"  Journal  of  the  .'nsoetatiou  for  Com- 
puting Maahinepg,  31-39,  1974. 

[10]  H.D.  Ni  and  J.K.  Agga>*v.'al , "Two-diinensional  filtering  and  its  error 

1 

analysis,"  IEEE  Trancaations  on  Computcvs,  C-23,  942-954,  1974. 

[11]  T.J.  Wagner,  "Nonparametric  estimates  of  probability  densities," 

^ .•  IEEE  Transactions  on  Infoimation  Theory,  IT-21 , 438-440,  1975. 

’ [12]  T.M.  Cover  and  T.J.  Wagner,  "Topics  in  statistical  pattern  recogni- 

,.i  tion,"  Digital  Patteim  Recognition,  K.S.  Fu,  Editor,  Springer-Vcrlag, 

New  York,  15-46,  1976. 

[13]  L.P.  Devroye,  "On  the  convergence  of  statistical  search,"  IEEE  Trans- 

. actions  on  Systems,  Man  and  Cybernetics,  SMC- 6,  46-56i  1976. 

[14]  L.P.  Devroye,  "Probabilistic  search  as  a strategy  selection  procedure," 
IEEE  Transactions  on  Systems,  Man  and  Cybernetics,  SMC-6,  315-321, 

1976. 

[15]  M.D.  Ni  and  J.K.  Aggarwal , "Error  analysis  of  two-dimensional  recur- 
sive digital  filters  employing  floating-point  arithmetic,"  IEEE 
Transactions  on  Computers,  C-25,  755-759,  1976. 

[16]  M.T.  Manry  and  J.K.  Aggarwal,  "The  design  of  multidimensional  FIR 
digital  filters  by  phase  correction,"  IEEE  Transactions  on  Circuits 
and  Systems,  CAS-23,  185-199,  1976. 


[17]  T.J.  Waqner  and  L.P,  Devrove.  "A  distribution-free  performance  bound 
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[18]  L.P.  Devroye,  "A  class  of  optimal  perfoimance-diroctecl  probabilistic 
automata, " lEKliJ  Tinncaatio^is  on  Suttenio,  Man  and  CuLcnmetios^  SfiCrS, 
777-784,  1976. 

[19]  C.S.  Penrod,  "Nonparametric  estimation  with  local  rules,"  Ph.D.  Thesis, 
University  of  Texas  at  Austin,  December  1976. 

[20]  L.P.  Devroye,  "Nonparametric  discrimination  and  density  estimation," 
Ph.D.  Thesis,  University  of  Texas  at  Austin,  December  1976. 

[21]  L.P.  Devroye,  "On  random  search  with  a learning  memory,"  Proceedings 
of  the  International  Conference  on  Cybernetics  and  Society,  Washing- 
ton, D.C.,  November  1976. 

[22]  T.J.  Wagner  and  L.P.  Devroye,  "The  strong  uniform  consistency  of  the 
nearest  neighbor  density  estimate,"  Annals  of  Statistics,  536-540, 
1977. 

[23]  T.J.  Wagner  and  C.S.  Penrod,  "Another  look  at  the  edited  nearest 
neighbor  rule,"  IEEE  Transactions  on  Systems,  Man  and  Cybernetics, 

SMC-7,  92-94,  1977. 

[24]  S.A.  Underwood  and  J.K.  Aggarwal , "Interactive  computer  analysis 
of  aerial  color  infrared  photographs,"  Computer  Graphics  and  Image 
Processing,  1-24. 

[25]  W.K.  Chow  and  J.K.  Aggarwal,  "Computer  analysis  of  planar  curvilinear 
moving  images,"  IEEE  Transactions  on  Conrputers,  C-26,  179-185,  1977. 

[26]  L.P.  Devroye  and  T.J.  Wagner,  "Distribution-free  performance  bounds 
with  the  resubstitution  error  estimate,"  Proceedings  of  the  IEEE 
Computer  Society  Conference  on  Pattern  Recognition  and  Image  Processing, 
Troy,  New  York,  323-326,  June  1977. 


[27]  L.P.  Devroye  and  T.J.  Wacjnor,  "Asymptotic  propotior,  of  clnsten'ncj 
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[40]  T.J.  Wagner  and  L.P.  Devroye,  "Generalized  kernel  estimates." 
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